Skip to content

Treasury module E2E tests#439

Open
dhirajs0 wants to merge 39 commits intoopen-web3-stack:masterfrom
dhirajs0:treasury-e2e-test
Open

Treasury module E2E tests#439
dhirajs0 wants to merge 39 commits intoopen-web3-stack:masterfrom
dhirajs0:treasury-e2e-test

Conversation

@dhirajs0
Copy link
Copy Markdown
Contributor

@dhirajs0 dhirajs0 commented Oct 9, 2025

End-to-end tests are added to cover the main flow of Treasury operations

  • Proposing a treasury spend
  • Approving and rejecting treasury proposals
  • Treasury payout execution
  • Handling proposal rejections and burns
  • Edge cases (e.g., insufficient funds, duplicate proposals)
  • Others

closes Issue #291

WIP

@dhirajs0 dhirajs0 marked this pull request as ready for review October 21, 2025 11:58
Comment thread packages/shared/src/treasury.ts
Comment thread packages/shared/src/treasury.ts Outdated
Copy link
Copy Markdown
Contributor

@github-actions github-actions Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The changes introduce a comprehensive suite of E2E tests for the treasury module. While the test coverage is good, there are several issues in packages/shared/src/treasury.ts that need attention, including commented-out code, duplicated logic, potentially brittle helpers, unsafe type assertions, and comments indicating some tests are not functional. The new test file for Kusama Asset Hub is well-structured.


Suggestions that couldn't be attached to a specific line

packages/shared/src/treasury.ts:509, 522

The claimTreasurySpend test contains comments (// Not working after moving to asset hub and // assert(dispatchedEvent) not getting the dispatch event...) that indicate the test is broken or incomplete. Incomplete tests should be fixed, marked as .skip, or removed before merging to avoid having non-functional tests in the suite.


packages/shared/src/treasury.ts:370-425, 489-562, 592-666

The test functions voidApprovedTreasurySpendProposal, claimTreasurySpend, and checkStatusOfTreasurySpend contain significant duplicated code for creating and verifying a spend proposal. This setup logic should be extracted into a reusable helper function to reduce code duplication and improve maintainability.

Comment thread packages/shared/src/treasury.ts Outdated

import { assert, expect } from 'vitest'

//import { logAllEvents } from './helpers/helper_functions.js'
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

A commented-out import //import { logAllEvents } from './helpers/helper_functions.js' is present. Unused and commented-out code should be removed to improve code clarity.

Comment on lines +117 to +118
const [event] = (await assetHubClient.api.query.system.events()).filter(
({ event }: any) => event.section === 'treasury' && event.method === eventType,
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The getSpendIndexFromEvent helper function filters for an event and directly accesses the first element of the result. This assumes only one event of the specified type will be present in the block's events. This could cause flaky tests if multiple such events are emitted. The function should be made more robust to handle cases where multiple events are found, or it should be documented why only one is expected. Additionally, the event is typed as any, which is not type-safe.

Comment thread packages/shared/src/treasury.ts Outdated
Comment on lines +502 to +503
// const initialSpendCount = await getSpendCount(assetHubClient)
const initialSpendCount = (await assetHubClient.api.query.treasury.spendCount()).toNumber()
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

There is a commented-out line of code (// const initialSpendCount = ...) which should be removed. Immediately after, the logic from the getSpendCount helper function is duplicated. You should use the existing getSpendCount helper for consistency and to avoid code duplication.

Comment thread packages/shared/src/treasury.ts Outdated
const pendingOrFailedSpends = spends.filter((spend) => {
const spendData = spend[1]?.unwrap()
return (
(spendData?.status.isPending || spendData?.status.isFailed) && // not pending or failed
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The comment // not pending or failed is inconsistent with the code (spendData?.status.isPending || spendData?.status.isFailed). The comment should be corrected to // is pending or failed to accurately reflect the logic.


// call payout tx for each pending or failed spend
for (const spend of pendingOrFailedSpends) {
const spendIndex = spend[0].toHuman?.() as number
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The code spend[0].toHuman?.() as number is an unsafe way to get the spend index from a StorageKey. toHuman() can return complex objects, and the type assertion as number may fail at runtime. Please use the safer method spend[0].args[0].toNumber() to extract the index.

// verify the spends status is attempted
for (const spendIndex of spendIndices) {
const spend = await assetHubClient.api.query.treasury.spends(spendIndex)
expect(spend?.unwrap()?.status.isAttempted).toBe(true)
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The expression spend?.unwrap()?.status.isAttempted can cause a runtime error if spend is None. unwrap() will be called on undefined and throw. You should first assert that the Option is Some before unwrapping it. For example: expect(spend.isSome).toBe(true); expect(spend.unwrap().status.isAttempted).toBe(true);.

@dhirajs0
Copy link
Copy Markdown
Contributor Author

All tests are passing for KAH locally.

@dhirajs0 dhirajs0 requested a review from rockbmb October 28, 2025 11:46
@dhirajs0
Copy link
Copy Markdown
Contributor Author

All the tests are now working on the PAH as well locally tested.

for (const spendIndex of spendIndices) {
const checkStatusEvents = await sendCheckStatusTx(assetHubClient, spendIndex)
await assetHubClient.dev.newBlock()
await verifyEventSpendProcessed(checkStatusEvents)
Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You might not want this here: from what I can tell, this verifyEventSpendProcessed helper matches against a snapshot; this is being done from within a loop, whose iteration count is unknown.
There could be 1 approved spend in the chain this is running on, or 1000.

For each iteration, a snapshot will be created, which might have already become obsolete by the time the next test run begins.

It is better to just inspect for the presence of the event without snapshotting it.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks. I got it. I will update it.

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I see you moved these tests to paritytech/ahm-dryrun#261, so this is not urgent.

However, for future reference, checking the same snapshot multiple times in one test is very often unintended.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, since I have already added the test in the ahm-dryrun repo. I will be removing it from here, as it will be redundant here.

Copy link
Copy Markdown
Contributor

@github-actions github-actions Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The new E2E tests for the treasury module are a valuable addition. However, several areas in the implementation could be improved to make the tests more robust, precise, and less brittle. The feedback focuses on strengthening assertions, improving type safety, and making tests less reliant on potentially fragile implementation details.

TInitStoragesPara extends Record<string, Record<string, any>> | undefined,
>(assetHubClient: Client<TCustom, TInitStoragesPara>, eventType: 'AssetSpendApproved' | 'Paid'): Promise<number> {
const [event] = (await assetHubClient.api.query.system.events()).filter(
({ event }: any) => event.section === 'treasury' && event.method === eventType,
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The use of any for the event record type weakens type safety. Please define or import a proper type for the event record to ensure type checking and improve code clarity.

TInitStoragesPara extends Record<string, Record<string, any>> | undefined,
>(assetHubClient: Client<TCustom, TInitStoragesPara>) {
await checkSystemEvents(assetHubClient, { section: 'treasury', method: 'AssetSpendApproved' })
.redact({ redactKeys: /expireAt|validFrom|index|data/ })
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Redacting the entire data field in the event snapshot is too broad and may hide potential regressions. It's better to be more specific and only redact fields within data that are non-deterministic (like timestamps or indices), while still capturing the overall structure and values of the event data.

// ensure that Alice's balance is increased
const balanceAfter = await assetHubClient.api.query.system.account(testAccounts.alice.address)
const balanceAmountAfter = balanceAfter.data.free.toBigInt()
expect(balanceAmountAfter - balanceAmountBefore).toBeGreaterThan(0n)
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The assertion expect(balanceAmountAfter - balanceAmountBefore).toBeGreaterThan(0n) is too weak. It only confirms that the balance has increased. For a more robust test, please assert that the balance has increased by the spendAmount, potentially accounting for transaction fees if they can be determined or estimated. A more precise check will better validate the payout logic.

// Ensure that Alice's balance is increased
const balanceAfter = await assetHubClient.api.query.system.account(testAccounts.alice.address)
const balanceAmountAfter = balanceAfter.data.free.toBigInt()
expect(balanceAmountAfter - balanceAmountBefore).toBeGreaterThan(0n)
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Similar to the comment for line 395, the assertion expect(balanceAmountAfter - balanceAmountBefore).toBeGreaterThan(0n) is not precise enough. The test should verify that the balance increase matches the expected spendAmount to ensure the correct amount was paid out.


// call payout tx for each pending or failed spend
for (const spend of pendingOrFailedSpends) {
const spendIndex = spend[0].toHuman?.() as number
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Using .toHuman() and casting to number to get the spendIndex is fragile and can break if the human-readable format changes. A more robust way is to get the index directly from the StorageKey arguments, for example: const spendIndex = (spend[0].args[0] as u32).toNumber();. This approach is more resilient to changes in the underlying library.

// verify the spends status is attempted
for (const spendIndex of spendIndices) {
const spend = await assetHubClient.api.query.treasury.spends(spendIndex)
expect(spend?.unwrap()?.status.isAttempted).toBe(true)
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The use of optional chaining (spend?.unwrap()?.status.isAttempted) can hide a potential test failure if spend is None. It would be better to first assert that the spend exists (expect(spend.isSome).toBe(true)) and then unwrap it. This makes the test fail explicitly if the spend is unexpectedly missing.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants